Assessing Artificial Neural Network Pruning Algorithms

In this study, three major pruning techniques are used to examine the effects of network pruning on classification accuracy. A feed-forward neural network structure which learns the characteristics of the training data via the backpropagation learning algorithm is employed to classify a multisource remotely sensed data set. An important conclusion from this study is that, of the three techniques tested, optimal brain surgeon (OBS) (the most sophisticated of the algorithms used) gave the best results. Results also show that pruning techniques are effective in that the pruned network is still capable of classifying the test data with high accuracy even after a considerable number of links have been removed. However, as a result of network pruning, the positive trend in overall accuracy may not apply to individual class accuracies. In order to appreciate the effects of pruning we have used visualisation techniques (including 3D visualisation and animations) and found them effective in providing insight into the process. In summary, this study shows that pruning algorithms are effective in reducing network size and producing a network that has a greater capacity for generalisation. This study also demonstrates the value of visualisation techniques in understanding the behaviour of artificial neural networks. In Proceedings of the 24 Annual Conference and Exhibition of the Remote Sensing Society, Greenwich, UK, pp. 603-609, 9-11 September 1998.